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University  Research  Initiative  Program 


Executive  Summary 


Introduction.  The  University  Research  Initiative  Program,  which  Congress 
mandated  in  1986,  is  one  element  of  the  DoD  basic  research  program.  The  Director  of 
Defense  Research  and  Engineering  provides  management  oversight  of  the  University 
Research  Initiative  Program.  The  Military  Departments  are  responsible  for  selecting, 
awarding,  and  administering  research  grants  issued  to  universities  and  colleges  under 
the  program. 

Audit  Objectives.  Our  audit  objective  was  to  determine  whether  policies  and 
procedures  for  selecting,  awarding,  and  administering  projects  under  the  University 
Research  Initiative  Program  are  effective.  We  also  evaluated  the  adequacy  of 
management  controls  related  to  the  audit  objective. 

Audit  Results.  The  Military  Departments'  policies  and  procedures  for  selecting, 
awarding,  and  administering  University  Research  Initiative  Program  grants  were 
generally  effective.  The  Navy  implemented  an  effective  management  control  program. 
However,  the  Army  Research  Office  did  not  adequately  implement  its  management 
control  program,  and  the  Air  Force  Office  of  Scientific  Research  did  not  establish  the 
required  management  control  program.  As  a  result,  the  Army  Research  Office  and  Air 
Force  Office  of  Scientific  Research  could  not  necessarily  determine  whetiier  their 
functions  complied  with  DoD  policies  and  procedures  and  whether  operations  were 
effective  and  efficient. 

Summary  of  Recommendations.  We  recommend  that  the  Director,  Army  Research 
Office,  revise  its  management  control  program  to  include  technical  oversight  of  grants 
as  a  mission-critical  function.  We  also  recommend  that  the  Director,  Air  Force  Office 
of  Scientific  Research,  establish  the  required  Air  Force  management  control  program 
and  provide  training  to  personnel  responsible  for  the  management  control  program. 

Management  Comments.  The  Director,  Army  Research  Office,  and  Deputy 
Director,  Air  Force  Office  of  Scientific  Research,  provided  comments  to  the  draft 
report.  The  Army  concurred  with  the  report  recommendation  and  stat^  that  an 
internal  process  action  team  would  review  and  report  on  the  existing  practices  in  die 
area  of  management  control.  The  Air  Force  concurred  with  the  report  recommendation 
and  stated  that  it  would  expand  its  current  management  control  program  and  provide 
training  where  needed.  Part  I  summarizes  management  comments,  and  Part  HI 
contains  the  complete  text  of  the  comments. 
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Part  I  -  Audit  Results 


Audit  Background 


The  DoD  basic  research  program  plays  a  critical  role  in  developing  technology 
and  in  educating  and  training  scientific  personnel  needed  to  support  continuing 
military  technological  advances.  The  University  Research  Initiative  Program, 
which  Congress  mandated  in  1986,  is  one  element  of  the  DoD  basic  research 
program. 

The  University  Research  Initiative  Program  funds  Defense  research  performed 
at  universities  and  colleges  as  well  as  supports  the  education  of  future  scientists 
and  engineers  in  disciplines  critical  to  national  Defense  needs.  The  University 
Research  Initiative  Program  supports  academic  researchers;  funds  the  studies  of 
graduate  and  undergraduate  students;  encourages,  through  joint  funding, 
collaborative  research  between  academia  and  industry;  assists  in  the 
procurement  of  research  equipment  necessary  to  conduct  the  research  at 
universities  and  colleges;  and  provides  research  funds  to  historically  minority 
colleges. 

The  Office  of  the  Director  of  Defense  Research  and  Engineering  provides 
overall  management  of  the  University  Research  Initiative  Program,  and  the 
Military  Departments  execute  the  program  for  DoD.  The  Office  of  the  Director 
of  Defense  Research  and  Engineering  overall  management  includes  the  review 
of  research  topics  proposed  for  funding  for  relevance  to  military  requirements. 
DoD  research  officials  convene  panels  to  select  military  research  projects. 
Oversight  panels  are  composed  of  representatives  from  the  Military 
Departments,  DoD  research  laboratories,  and  private  industry. 

The  Military  Departments  provide  funding  under  the  University  Research 
Initiative  Program  through  grants  using  the  policies  in  Office  of  Management 
and  Budget  Circular  A- 110,  "Uniform  Administrative  Requirements  for  Grants 
and  Agreements  With  Institutions  of  Higher  Education,  Hospitals  and  Other 
Non-Profit  Organizations,"  November  29,  1993.  The  Circular  A- 110  policies 
are  supplemented  in  interim  guidance  in  DoD  Regulation  3210. 6-R,  "DoD 
Grant  and  Agreement  Regulations,"  February  4,  1994.  The  contract  officials  at 
the  Military  Departments'  research  offices  negotiate,  award,  and  administer 
University  Research  Initiative  Program  grants  using  the  guidance  in 
Circular  A-1 10  and  Regulation  3210. 6-R. 
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Audit  Results 


Audit  Objectives 


The  primary  audit  objective  was  to  determine  whether  policies  and  procedures 
for  selecting,  awarding,  and  administering  projects  under  the  University 
Research  Initiative  Program  are  adequate.  We  also  evaluated  the  adequacy  of 
management  controls  related  to  the  primary  audit  objective.  Appendix  A 
describes  the  audit  scope  and  methodology. 


Administering  the  University  Research  Initiative  Program 


Based  on  the  results  of  our  audit,  the  University  Research  Initiative  Program 
was  generally  effective  in  the  following  areas: 

o  DoD  is  properly  advertising  research  interests  for  receipt  of  research 
proposals.  Upon  receipt  of  research  proposals,  DoD  has  an  effective  process  of 
convening  panels  for  review  and  selection  of  proposals. 

o  The  Military  Departments  awarded  grants  issued  under  the  program  to 
the  universities  and  colleges  in  accordance  with  established  policies  and 
procedures. 

0  the  Military  Departments'  technical  representatives  generally 
administered  grants  well. 

Appendix  B  discusses  in  more  detail  the  audit  results  in  the  selection,  award, 
and  administration  of  grants  under  the  University  Research  Initiative  Program. 
However,  implementation  of  management  control  programs  needs  improvement 
and  is  discussed  in  the  finding. 
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Management  Control  Programs 

The  Army  Research  Office  did  not  adequately  implement  its 
management  control  program,  and  the  Air  Force  Office  of  Scientific 
Research  (AFOSR)  did  not  establish  the  required  management  control 
program.  The  Anny  did  not  have  an  adequate  management  control 
program  because  the  Army  Research  Office  did  not  ensure  review  of  all 
areas  critical  to  die  accomplishment  of  its  mission.  AFOSR  did  not 
properly  emphasize  the  importance  of  management  controls  and  did  not 
provide  adequate  training  to  responsible  personnel.  As  a  result,  the 
Army  Research  Office  and  AFOSR  could  not  necessarily  determine 
whether  their  functions  complied  with  DqD  policies  and  procedures  and 
whether  operations  were  effective  and  efficient. 


Management  Controls  and  Policy 


Managers  are  responsible  for  the  quality  and  timeliness  of  the  Federal  progr^s 
and  operations  diat  they  manage,  for  increasing  programs  and  operations 
productivity,  and  for  controlling  program  and  operation  costs.  Management 
controls  are  the  organization's  policies  and  procedures  used  to  ensure  that  the 
program  achieves  the  intended  results;  that  resources  are  used  consistently  with 
the  organization's  mission  and  in  accordance  with  laws  and  regulations;  and  that 
reliable  and  timely  information  is  obtained,  maintained,  reported,  and  used  for 
decisionmaking.  Management  controls  are  not  stand-alone  activities  but  are 
integrated  into  the  day-to-day  operational  responsibilities  of  the  organization 
managers. 

The  Office  of  Management  and  Budget  established  management  control  policy 
by  issuing  Circular  A- 123,  and  DoD  supplemented  the  policy  by  issuing 
directives,  regulations,  and  instructions.  The  Office  of  Management  and  Budget 
revised  management  control  policy  in  June  1995,  and  DoD  officials  revised 
DoD  management  control  policy  in  August  1996.  This  report  discusses  both  the 
revised  guidance  and  previous  guidance,  because  the  grants  reviewed  were 
awarded  during  FYs  1994  through  1996.  Army  Research  Office  and  AFOSR 
officials  used  the  earlier  versions  during  the  audit  timeframe.  However,  we 
considered  the  new  guidance  when  making  our  recommendations  for  corrective 
action. 

Office  of  Management  and  Budget  Policy.  Office  of  Management  and  Budget 
Circular  A- 123,  Revised,  "Management  Accountability  and  Control,"  June  21, 
1995,  replaced  Office  of  Management  and  Budget  Circular  A-123,  Revised, 
"Internal  Control  Systems,"  August  4,  1986,  and  Office  of  Management  and 
Budget  "Internal  Controls  Guidelines,"  December  1982.  The  new  Circular 
provides  greater  flexibility  by  giving  agencies  the  discretion  to  determine 
procedures  to  use  in  the  proper  stewardship  of  Federal  resources.  The  new 
Circular  does  not  require  agencies  to  institute  a  separate  management  control 
process,  and  it  gives  agencies  the  discretion  to  determine  what  procedures  to  use 
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Management  Control  Programs 


in  establishing,  assessing,  correcting,  and  reporting  on  management  controls. 
The  Circular  states: 

Agencies  and  individual  Federal  managers  must  take  systematic  and 
proactive  measures  to  (i)  develop  and  implement  appropriate,  cost- 
effective  management  controls  for  results-oriented  management;  (ii) 
assess  the  adequacy  of  management  controls  in  Federal  programs  and 
operations;  (iii)  identify  needed  improvements;  (iv)  take 
corresponding  corrective  action;  and  (v)  report  annuity  on 
management  controls. 

DoD  Management  Control  Policy.  DoD  implemented  the  management  control 
policy  by  issuing  DoD  Directive  5010.38,  "Internal  Management  Control 
Program,"  April  14,  1987,  and  DoD  Instruction  5010.40,  "Management 
Control  (MC)  Program  Procedures,"  August  28,  1996.  DoD  Directive  5010.38 
requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls.  DoD 
Directive  5010.38,  "Management  Control  (MC)  Program,"  Au^st26,  1996, 
cancels  the  earlier  directive  and  requires  the  continuous  monitoring  of  and 
improving  the  effectiveness  of  management  controls.  The  revised  guidance 
does  not  change  management's  responsibility  to  design  management  structures 
that  help  ensure  accountability  for  results  and  to  include  appropriate,  cost- 
effective  management  controls.  Management  control  objectives  include 
complying  with  applicable  laws;  safeguarding  assets;  properly  accounting  for 
revenues  and  expenditures;  and  avoiding  fraud,  waste,  and  mismanagement. 

DoD  Instruction  5010.40  prescribes  procedures  for  implementation  and  use  of 
management  control  programs.  DoD  policy  requires  that  DoD  organizations 
establish  management  control  programs  to  review,  evaluate,  and  report  on  the 
effectiveness  of  management  controls.  In  addition,  DoD  management  is 
required  to  take  appropriate  action  to  identify  and  correct  ineffective 
management  controls  and  to  establish  management  controls  when  warranted. 


Army  Management  Control  Program 

Army  Policy.  The  Army  revised  Army  Regulation  11-2,  "Management 
Control,"  on  August  1,  1994.  The  Army  regulation  streamlines  the 
management  control  process,  reinforces  the  accountability  of  Army  commanders 
and  managers  for  establishing  and  maintaining  effective  management  controls, 
and  provides  Army  commanders  and  managers  with  greater  flexibility  in  their 
evaluation  of  the  controls.  The  revised  regulation  decentralizes  the  formulation 
of  a  management  control  plan.  The  Army  Management  Control  Program  is  a 
written  plan  for  conducting  required  management  control  evaluations  within  the 
assessable  unit  over  a  5-year  period.  Under  the  old  regulation,  the  Army 
Materiel  Command  published  a  single  management  control  plan,  which  the 
regulation  encouraged,  but  did  not  require,  field  organizations  to  supplement  to 
address  command-unique  or  location-unique  circumstances.  Under  the  new 
regulation,  major  subordinate  commands  and  separate  reporting  organizations. 
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such  as  the  Army  Research  Office,  will  prepare  their  own  management  control 
plans. 

Army  Research  Office  Management  Control  Program.  Army  Research 
Office  officials  did  not  adequately  implement  its  management  control  program 
because  the  program  did  not  include  a  review  of  all  mission-critical  functions  to 
accomplish  its  mission.  Instead,  the  Army  Research  Office  continued  to  use  the 
Army  Materiel  Command  management  control  plans  for  FYs  1995  and  1996. 
The  plans  included  evaluations  of  functional  areas  for  procurement,  budget 
execution,  information  management  control,  supply  and  property  management, 
and  personnel  management.  Although  those  functional  areas  are  important  to 
accomplish  the  Army  Research  Office's  mission,  die  Army  Research  Office  did 
not  supplement  the  plan  to  include  technical  oversight  of  grants.  The  Army 
Research  Office  needs  to  include  grant  technical  oversight  as  a  mission-critical 
function  in  the  management  control  plan  to  ensure  that  the  Army  Research 
Office  achieves  its  mission  of  fostering  and  maintaining  an  Army  science  base 
and  to  ensure  that  grantees  are  properly  complying  with  the  conditions  of  the 
grant.  An  example  of  the  lack  of  grant  monitorship  is  illustrated  with  a 
University  of  Kentucky  grant. 

University  of  Kentucky.  The  Army  Research  Office  technical  monitor  for  a 
University  of  Kentucky  augmentation  grant,  DAAH04-95-1-0357,  was  unaware 
that  the  principal  investigator  responsible  for  the  performance  of  the  research  on 
the  parent  grant  had  resigned  from  the  University,  and  no  research  was 
performed  under  the  grant.  The  Army  Research  Office  officials  awarded  the 
grant  to  the  University  of  Kentuc^  on  May  19,  1995,  and  provided 
three  quarterly  scheduled  payments  totaling  $34,998.  The  Army  Research 
Office  discontinued  the  quarterly  payments  upon  notification  firom  the 
University  of  Kentucky  that  the  principal  investigator  transferred  to  another 
university.  The  Army  Research  Office  technical  monitor  should  have  been 
aware  that  the  principal  investigator  transferred  and  that  no  research  had  been 
performed,  and  should  have  initiated  termination  of  the  grant. 

The  Army  Research  Office  is  responsible  for  its  own  management  control 
program  and  for  development  of  the  associated  management  control  plan 
identifying  functional  areas  for  review.  The  Army  Research  Office 
management  control  plan  should  include  grant  technical  oversight  as  a  mission- 
critical  function. 


Air  Force  Management  Control  Program 


Air  Force  Policy.  Air  Force  Policy  Directive  65-2,  "Internal  Management 
Controls,"  March  10,  1993,  states  that  "Commanders  and  managers  at  every 
level  of  the  Air  Force  are  responsible  for  establishing,  evaluating,  improving, 
and  reporting  on  internal  controls." 

Air  Force  Instruction  65-201,  "Internal  Management  Control  Program,"  July  1, 
1994,  implements  Air  Force  Policy  Directive  65-2.  The  instruction  provides  a 
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structure  to  establish,  evaluate,  and  report  on  the  Air  Force  management 
controls  in  all  functional  areas.  The  instruction  states,  "Managers  must  apply 
controls  to  all  administrative  and  programmatic  functions. " 

Air  Force  Instruction 65-201 /Air  Force  Materiel  Command  Supplement!, 
"Internal  Management  Controls  Program,"  April  17,  1995,  implements 
Air  Force  Instruction  65-201 .  The  instruction  elaborates  on  the  Air  Force 
Materiel  Command's  role  in  supporting  the  Air  Force  Management  Control 
Program  and  is  not  intended  as  a  stand-alone  instruction.  Air  Force 
Instruction  65-201  requires  AFOSR  management  to  establish  and  evaluate 
management  controls  and  to  prepare  a  management  control  plan  and  an  annual 
statement  of  assurance.  A  management  control  plan,  updated  annually,  lists  the 
inventory  of  assessable  units  and  schedules  vulnerability  assessments  and 
management  control  reviews.  The  annual  statement  of  assurance  states  whether 
or  not  management  has  reasonable  assurance  that  the  objectives  of  the 
management  control  program  are  met.  A  management  control  review  is  a  tool 
used  to  evaluate  the  organizational  management  controls. 

AFOSR  Regulation  123-1,  "Self-Inspection  Program,"  February  1,  1988, 
establishes  self-inspection  program  requirements  for  all  AFOSR  personnel 
managing  functional  areas.  The  self-inspection  program  enables  the 
Commander  and  supervisors  at  all  management  levels  to  identify  and  correct 
management  deficiencies  and  noncompliance  with  directives.  The  regulation 
outlines  responsibilities,  provides  procedures  for  developing  and  using 
checklists,  and  directs  aimual  self-inspections. 

Air  Force  Office  of  Scientific  Research  Management  Control  Program. 
AFOSR  management  did  not  establish  a  management  control  program  as 
required  by  Air  Force  policy.  Also,  the  AFOSR  official  responsible  for  the 
management  control  program  had  limited  training  in  establishing  an  effective 
program.  Although  AFOSR  submitted  the  annual  statements  of  assurance  for 
FYs  1995  and  1996  to  the  Air  Force  Materiel  Command  as  required,  the  annual 
statements  were  not  based  on  an  examination  of  functions  at  the  organization  as 
outlined  in  Air  Force  Instraction  65-201 . 

AFOSR  is  responsible  for  supporting  research  effo^  that  are  of  interest  to  the 
Air  Force  at  universities  and  colleges.  The  support  includes  providing  funds  for 
undergraduate  and  graduate  studies,  providing  support  for  domestic  and 
international  research  programs,  and  providing  support  for  the  acquisition  of 
research  resources. 

The  lack  of  a  management  control  program  and  the  associated  management 
control  plan  has  inhibited  AFOSR  management  from  identifying  functions  that 
require  attention  as  illustrated  in  the  following  example  of  the  lack  of  controls  in 
the  receipt  and  processing  of  interest  checks  associated  with  grant  funding. 

University  of  Southern  California  Interest  Checks.  Office  of  Management 
and  Budget  Circular  A- 110  requires  grantees  to  deposit  advanced  funds  in 
interest-bearing  accounts  and  submit  earned  interest  to  the  Government.  We 
examined  the  AFOSR  process  for  receiving  and  processing  interest  checks  and 
found  the  controls  to  be  inadequate.  Although  AFOSR  implemented  an 
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Air  Force  Audit  Agency  recommendation  for  establishing  procedures  securing 
checks  in  a  secure  container  and  procedures  for  transporting  the  checks  for 
deposit,  the  proeedures  did  not  address  other  safeguards  such  as  recording  the 
check  receipt  in  a  log,  controlling  documentation  necessary  for  check  deposit, 
and  forwarding  the  interest  cheeks  for  deposit  in  a  timely  manner. 

The  University  of  Southern  California  submitted  37  interest  checks,  valued  at 
$105,400,  on  a  monthly  basis  to  AFOSR  between  January  21,  1993,  and 
December  11,  1995.  Although  AFOSR  personnel  provided  documentation  of 
the  transfer  of  interest  checks  to  the  Defense  Finance  and  Accounting  Service,  it 
took  AFOSR  personnel  6  weeks  to  provide  the  transfer  documentation  after 
extensive  research.  In  addition,  AFOSR  did  not  process  interest  checks  for 
deposit  in  a  timely  manner,  and  the  processing  of  interest  checks  for  deposit 
ranged  from  2  weeks  to  4  montiis. 

Although  AFOSR  had  some  management  control  procedures  in  various 
functional  areas,  the  establishment  of  a  management  control  program  at  AFOSR 
could  have  identified  inadequate  procedures  for  the  receipt  of  interest  checks. 
AFOSR  lacked  an  effective  overarching  management  control  program  because 
AFOSR  management  did  not  emphasize  policy  requirements  and  did  not  ensure 
that  the  personnel  responsible  for  the  management  control  program  received  the 
necessary  training. 


Navy  Management  Control  Program 

The  Office  of  Naval  Research  implemented  an  effective  management  control 
program  because  it: 

o  identified  mission-critical  functional  areas, 

0  had  documentation  to  identify  that  functional  areas  were  reviewed  for 
compliance  with  policies  and  procedures, 

o  identified  functional  area  weaknesses  and  took  corrective  actions,  and 

o  properly  supported  the  annual  statement  of  assurance. 


Conclusion 


The  DoD  Management  Control  Program  (MCP)  entails  risk  assessment, 
periodic  self  evaluation  and  reporting  of  material  control  weaknesses. 
Compliance  wife  the  requirements  of  die  MCP  is  necessary  for  obtaining 
reasonable  assurance  that  the  Army  Research  Office  and  AFOSR  accomplish 
programs  and  functions  as  intended  and  that  they  provide  reliable  and  timely 
information  to  decisionmakers.  The  Army  Research  Office  and  AFOSR  should 
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integrate  management  control  evaluation  measures  with  other  management 
policies,  regulations,  and  procedures.  The  Army  Research  Office  and  AFOSR 
management  controls  should  be  consistent  with  the  organizations'  unique 
mission  of  supporting  research  and  development  efforts.  Copies  of  this  report 
should  be  provided  to  the  senior  Army  and  Air  Force  MCP  officials. 


Recommendations  for  Corrective  Action 


1.  We  recommend  that  the  Director,  Army  Research  Office,  modify  its 
management  control  program  to  include  the  grant  technical  oversight  function  as 
a  mission-critical  function. 

2.  We  recommend  that  the  Director,  Air  Force  Office  of  Scientific  Research, 
establish  a  management  control  pro^am  as  required  by  Air  Force  policy  and 
provide  training  to  personnel  responsible  for  management  controls. 


Management  Comments 


Army  Comments.  The  Army  Research  Office  concurred  with  the 
recommendation  to  modify  its  management  control  program  and  plans  to 
establish  a  total  quality  management  process  action  team  to  review  and  report  on 
the  existing  practices  and  to  recommend  improvements  upon  issuing  its  final 
report  August  25,  1997. 

Air  Force  Comments.  The  Air  Force  Office  of  Scientific  Research  concurred 
with  the  recommendation  to  establish  a  management  control  program  and 
provide  training  where  needed,  and  plans  to  expand  the  current  management 
control  program  for  Management  of  Science  and  Technology. 
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Part  II  -  Additional  Information 


Appendix  A.  Audit  Process 


Scope 


We  performed  the  economy  and  efficiency  audit  from  September  1996  through 
March  1997.  To  accomplish  the  audit  objectives,  we  judgmentally  selected  and 
reviewed  89  grants,  valued  at  $22  million,  issued  by  the  Military  Departments 
to  9  universities,  during  FYs  1994  through  1996.  We  did  not  review  the  merits 
of  selecting  the  research  grants  because  of  the  scientific  nature  of  the  research, 
but  we  did  review  the  overall  process  for  selecting  research  grants.  We 
examined  the  Military  Departments'  policies  and  procedures  for  awarding  and 

administering  University  Research  Initiative  Program  grants. 

We  also  reviewed  the  records  of  4  of  the  9  universities  in  relation  to  the 
expenditure  rates  of  advanced  research  funds  and  the  submission  of  earned 
interest  to  the  Government.  Appendix  C  identifies  the  grants  reviewed. 


Audit  Period  and  Standards 


We  performed  this  economy  and  efficiency  audit  from  September  1996  through 
March  1997  in  accordance  with  auditing  standards  issued  by  the  Comptroller 
General  of  the  United  States,  as  implemented  by  the  Inspector  General,  DoD. 
We  included  tests  of  management  controls  considered  necessary. 


Methodology 

We  reviewed  policies  and  procedures  for  awarding  and  administering  University 
Research  Initiative  Program  grants  at  the  Military  Departments,  examined 
selected  grants,  and  conducted  interviews  with  Military  Department  and 
academic  personnel.  Specifically,  we: 

o  reviewed  research  and  development  grant  policies  and  procedures 
dated  July  1976  through  May  1996; 

0  reviewed  the  Military  Departments'  research  grant  files  and 
correspondence  dated  1994  through  1996; 

0  interviewed  Military  Departments'  technical  directors  and  technical 
managers  and  the  Services'  grant  officials; 
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o  interviewed  administrative  grant  officers; 

o  interviewed  academic  personnel  at  four  universities;  and 

o  reviewed  accounting  records  dated  May  1994  through  January  1997  at 
four  universities. 


Use  of  Computer-Processed  Data 


We  relied  on  computer-processed  data  from  the  Military  Departments'  data 
bases  without  testing  the  system's  general  and  application  controls  to  confirm 
the  reliability  of  the  data.  However,  before  using  &e  data  bases  to  represent  the 
University  Research  Initiative  Program  universe  and  selecting  a  sample,  we 
performed  tests  to  determine  the  completeness  of  the  data  bases.  In  addition, 
we  did  not  identify  any  significant  inaccuracies  in  the  information  obtained  from 
the  Military  Departments'  data  bases  while  performing  the  audit. 


Prior  Audits  and  Other  Reviews 


No  audits  or  reviews  were  performed  in  the  last  5  years  on  the  University 
Research  Initiative  Program. 

Air  Force  Audit  Agency  Installation  Report  No.  50296028,  "Administration  of 
Advance  Payment  Pool  Agreements,  Air  Force  Office  of  Scientific  Research 
(AFOSR),"  March  12,  1996,  recommended  that  the  Director  ensure  the 
development  of  management  controls  for  the  processing  of  interest  checks 
associated  with  advanced  payments.  The  AFOSR  management  concurred  with 
the  recommendation  and  implemented  controls  that  required  the  securing  of  the 
interest  checks  in  a  secure  container  and  safeguarding  the  transporting  of  the 
checks  for  deposit. 


Organizations  and  Individuals  Visited  or  Contacted 


We  visited  or  contacted  individuals  and  organizations  within  the  DoD  and  the 
University  of  Kentucky,  Purdue  University,  University  of  California  at  San 
Diego,  and  University  of  Southern  California.  Further  details  are  available 
upon  request. 
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Appendix  B.  Audit  Objective  Results 

During  this  audit,  we  judgmentally  selected  and  reviewed  89  grants,  valued  at 
$22  million,  at  9  different  universities  (Appendix  C).  The  89  grants  were 
issued  by  either  the  Army  Research  Office,  die  Office  of  Naval  Research,  or 
AFOSR.  Although  we  identified  deficiencies  in  administering  of  individual 
grants,  the  deficiencies  were  not  systemic  of  an  overall  problem.  The  overall 
results  of  our  review,  by  objective,  are  discussed  below. 

Selecting  Research  Efforts.  The  selection  process  for  research  efforts  involves 
the  convening  of  scientific  panels  to  review  and  rate  research  proposals  that 
universities  and  colleges  submit  in  response  to  DoD  requests  for  research 
proposals.  The  scientific  panels  are  composed  of  officials  from  the  Office  of 
the  Secretary  of  Defense,  the  Military  Departments,  and  in  some  cases  private 
industry.  Our  review  of  the  process  of  advertising  and  selecting  research  efforts 
identified  no  significant  problems. 

Awarding  University  Research  Initiative  Program  Grants.  The  Military 
Departments  were  generally  awarding  the  University  Research  Initiative 
Program  grants  in  compliance  with  established  policies  and  procedures  of  the 
Office  of  Management  and  Budget  Circular  and  DoD  regulations.  The  grants 
reviewed  were  issued  using  die  guidelines  established  in  the  Office  of 
Management  and  Budget  circular  A-110,  DoD  Regulation  3210.6-R,  or  agency- 
specific  regulations.  In  addition,  grant  officers  were  trained  in  the  requirements 
for  awarding  grants.  The  audit  found  some  minor  irregularities  in  the  awarding 
process;  however,  the  irregularities  were  not  deemed  systemic.  For  example, 
documentation  of  the  analysis  of  grantees'  cost  proposals  could  be  improved, 
ensuring  that  they  properly  verified  all  the  labor  rates. 

Administering  University  Research  Initiative  Program  Grants.  In  general, 
the  Military  Departments'  technical  representatives  adequately  administered  the 
University  Research  Initiative  Program  grants  reviewed.  The  technical 
representatives  were  aware  of  the  requirements  of  the  grants  regarding 
monitoring  the  research  through  annual  progress  reports.  In  some  cases,  the 
researchers  were  not  submitting  research  progress  reports  in  a  timely  manner. 
However,  Military  Departments'  technical  representatives  used  other  methods  to 
monitor  the  progress  of  research  efforts,  including  the  review  of  technical 
papers  for  publication  concerning  the  research,  attendance  at  seminars  and 
symposiums  in  the  research  topic  area,  and  participation  in  workshops 
discussing  the  research  status.  In  addition,  examination  of  the  technical 
representatives'  performance  plans  identified  that  monitoring  research  efforts 
was  a  job  performance  element.  For  the  89  grants  reviewed  during  the  audit, 
the  administrative  responsibilities  and  duties  were  deemed  sufficient. 
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Armv  Grants 

Value  of 

Grant  Number 

Grantee  -  University 

Grant 

DAAH04-94-G-0083 

Duke 

$  10,787 

DAAH04-94-G-0174 

Duke 

99,163 

DAAH04-95-1-0243 

Duke 

100,000 

DAAH04-95-1-0275 

Duke 

89,999 

DAAH04-96-1-0246 

Duke 

275,000 

DAAH04-96-1-0283 

Duke 

51,736 

DAAH04-96-1-0448 

Duke 

431,000 

DAAH04-94-G-0110 

Purdue 

51,030 

DAAH04-94-G-0181 

Purdue 

84,724 

DAAH04-94-G-0210 

Purdue 

95,000 

DAAH04-94-G-0297 

Purdue 

96,540 

DAAH04-94-G-0365 

Purdue 

168,670 

DAAH04-95-1-0008 

Purdue 

150,000 

DAAH04-95-1-0010 

Purdue 

95,720 

DAAH04-95-1-0237 

Purdue 

100,000 

DAAH04-95-1-0246 

Purdue 

1,140,000 

DAAH04-95-1-0385 

Purdue 

51,709 

DAAH04-95-1-0513 

Purdue 

88,222 

DAAH04-96-1-0222 

Purdue 

161,764 

DAAH04-96-1-0331 

Purdue 

1,881,500 

DAAH04-96-1-0444 

Purdue 

3,000,000 

DAAH04-94-G-0105 

Kentucky 

61,660 

DAAH04-94-G-0243 

Kentucl^ 

172,300 

DAAH04-94-G-0331 

Kentucl^ 

309,685 

DAAH04-94-G-0344 

Kentucl^ 

329,061 

DAAH04-95-1-0357 

Kentucl^ 

140,000 

DAAH04-96- 1-0327 

Kentucky 

496,266 

DAAH04-96-1-0398 

Kentucky 

143,827 

DAAH04-96-1-0399 

Kentucky 

228.840 

Total:  29 

$10,104,203 
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Grant  Number 


N00014-95-1-0753 

N00014-95-1-0895 

N00014-95-1-1036 

N00014-95-1-1061 

N00014-95-1-1104 

N00014-95-1-1170 

N00014-95-1-1227 

N00014-96-1-0933 

N00014-96-1-0957 

N00014-96-1-1009 

N00014-96-1-1101 

N00014-94-1-0657 

N00014-94-1-0822 

N00014-94-1-0997 

N00014-94-1-1064 

N00014-95-1-0795 

N00014-95-1-0814 

N00014-95-1-0882 

N00014-95-1-0996 

N00014-95-1-1002 

N00014-96-1-1046 

N00014-96-1-1220 

N(X)014-94-l-0707 

N00014-94-1-0757 

N00014-95-1-0885 

N00014-95-1-1131 

N00014-95-1-1137 

N00014-95-1-1140 

N00014-96-1-0923 

Total:  29 


Naw  Grants 


Grantee  -  University 

Value  of 
Grant 

Oregon  State 

$  120,763 

Oregon  State 

110,141 

Oregon  State 

118,158 

Oregon  State 

299,575 

Oregon  State 

500,000 

Oregon  State 

200,000 

Oregon  State 

150,947 

Oregon  State 

122,225 

Oregon  State 

120,493 

Oregon  State 

154,490 

Oregon  State 

124,798 

California  at  San  Diego 

114,632 

California  at  San  Diego 

79,964 

California  at  San  Diego 

76,435 

California  at  San  Diego 

72,900 

California  at  San  Diego 

111,464 

California  at  San  Diego 

90,000 

California  at  San  Diego 

80,912 

California  at  San  Diego 

145,000 

California  at  San  Diego 

113,500 

California  at  San  Diego 

394,000 

California  at  San  Diego 

134,326 

Texas  at  Austin 

174,509 

Texas  at  Austin 

204,745 

Texas  at  Austin 

177,625 

Texas  at  Austin 

111,797 

Texas  at  Austin 

400,000 

Texas  at  Austin 

165,000 

Texas  at  Austin 

95.000 

$4,763,399 
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Air  Force  Grants 


Grant  Number 

Grantee  -  University 

Value  of 
Grant 

F49620-94-1-0327 

Princeton 

$  177,811 

F49620-94-1-0372 

Princeton 

196,691 

F49620-94-1-0389 

Princeton 

149,601 

F49620-94-1-0391 

Princeton 

153,001 

F49620-95-1-0111 

Princeton 

199,700 

F49620-95-1-0438 

Princeton 

139,044 

F49620-95-1-0532 

Princeton 

499,999 

F49620-96-1-0191 

Princeton 

150,000 

F49620-96-1-0261 

Princeton 

126,070 

F49620-96-1-0405 

Princeton 

305,165 

F49620-96-1-0511 

Princeton 

220,000 

F49620-94-1-0406 

California  at  Los  Angeles 

127,393 

F49620-95-1-0405 

California  at  Los  Angeles 

152,874 

F49620-95-1-0414 

California  at  Los  Angeles 

108,245 

F49620-95-1-0429 

California  at  Los  Angeles 

224,474 

F49620-95-1-0453 

California  at  Los  Angeles 

94,637 

F49620-95-1-0491 

California  at  Los  Angeles 

199,070 

F49620-95-1-0534 

California  at  Los  Angeles 

499,999 

F49620-96-1-0296 

California  at  Los  Angeles 

102,099 

F49620-96-1-0302 

California  at  Los  Angeles 

144,390 

F49620-96-1-0453 

California  at  Los  Angeles 

379,795 

F49620-96-1-0474 

California  at  Los  Angeles 

133,500 

F49620-94-1-0358 

Southern  California 

138,715 

F49620-94-1-0407 

Southern  California 

93,924 

F49620-95-1-0035 

Southern  California 

891,100 

F49620-95-1-0089 

Southern  California 

220,000 

F49620-95-1-0103 

Southern  California 

82,842 

F49620-95-1-0420 

Southern  California 

100,983 

F49620-95-1-0445 

Southern  California 

131,730 

F49620-95-1-0450 

Southern  California 

134,999 

F49620-95-1-0452 

Southern  California 

900.000 

Total:  31 

Total  University  Research  Initiative  Grants:  89 

$7,177,851 

$22,045,453 

Appendix  D.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Director,  Defense  Logistics  Studies  Information  Exchange 
Director  of  Defense  Research  and  Engineering 
Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Army 
Commander,  Army  Materiel  Command 
Director,  Army  Research  Office 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Chief  of  Naval  Research,  Office  of  Naval  Research 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 
Commander,  Air  Force  Materiel  Command 
Director,  Air  Force  Office  of  Scientific  Research 
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Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 


Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 

House  Subcommittee  on  Government  Management,  Iirformation,  and  Technology, 
Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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Part  ni  -  Management  Comments 


Department  of  the  Army  Comments 


DEPAimymWT  OF  THE  AMI  Y 
AFMV  PCSEARCH  OFFICE 
PAiOXmtl 

RESEARCH  TRIANGLE  PARK.  NC  27700^11 

imrcrjo 

ATTCNIIONOi' 

AMXRO-D 


MEiiORAKDUM  FOR  Auditor  General,  U.S.  Army  Audit  Agency,  3101  Park  Center  Drive, 
Alexandria,  VA  22302-1596 

SUBJECT;  Audit  Report  on  University  Reeearch  Initiative  Program  (Pro:)ect.  No.  6AB 
0077) 

1.  As  requeatcd  in  your  memorandum  dated  30  May  1997,  this  office  has  reviewed 
the  subject  draft  audit  report.  The  report  contends  that  ARO  ».,.did  not 
adequately  in^loment  its  management  control  program  because  it  did  not  include 
review  of  all  mission-critioal  functions  to  accomplish  its  mission." 

2.  The  report  epeclfioally  cites  the  need  to  Include  technical  oversight  of 
grants  as  a  mission-critical  function  In  the  management  control  plan  and 
references  a  University  of  Kentucky  (UK)  AA5BRT  grant  ae  an  example  of  inadaquaco 
technical  oversight .  The  UK  AASERT  grant  was  awarded  for  BMDO  to  augment  a  parent 
grant  being  monitored  by  the  Air  Force.  The  ARO  technical  monitor  was  unaware 
that  the  principal  Investigator  (PI)  reaponaible  for  the  performance  of  the 
research  under  the  parent  grant  resignad  from  the  University  and  performed  no 
research.  ARO  awarded  the  grant  to  UK  on  19  May  1997  and  bagan  providing 
scheduled  research  funding  support  on  19  Jul  1995.  The  PI  of  the  parent  grant 
resigned  from  the  Untvereity  of  Kentucky  in  October  1995.  ARO  provided  three 
quarterly  payments  for  the  AASERT  reeearch  effort  totalling  $34,990.  ARO 
discontinued  the  quarterly  payment  upon  notifioetion  from  UK  that  the  PI 
transferred  to  another  university  end  subsequently  recovered  all  payments. 

3.  ARO  has  a  long  history  of  awarding  and  administering  research  acquisition  and 
assistance  agreements.  Ne  believe  that  our  management  control  system  is  generally 
adequate  to  protect  the  interests  of  the  Army;  however,  we  recognixe  the  value  of 
independent  reviews  of  operating  practices  to  test  and  strengthen  these  systems. 

In  addressing  the  reported  deficiency  of  the  audit  report,  ARO  will  take 
corrective  action.  In  order  to  strengthen  the  technical  oversight  of  our  research 
contracts  and  grants.  X  hava  directed  that  a  TQH  Procass  Action  Team  (PAT)  be 
eetabliehed  to  review  our  cxieting  practices  in  this  area  and  recommend 
improvements  as  necessary.  The  PAT  will  be  headed  by  Dr.  George  Neece,  Director. 
Research  and  Technology  Integration.  Dr.  Neece*#  in-progress  report  ic  due  26 
July  1997;  the  final  report  is  due  25  August  1997. 

4.  POC  for  this  action  is  Mr.  Jack  1#.  Harless,  Dirsetor,  Resources  end 
Acquisition  Management.  He  can  be  reached  at  (919)  549-4239  or  D5K  632-4239. 


Copy  Furnished: 
OASA(ROA)  (Ms.  Kominos) 
AMCRM 
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department  of  the  air  force 

AIR  FORCE  OFFICE  OF  SCIERTIFIC  RESEARCH  (AFMC) 


1  JUL  97 


1947  -  1997 


memorandum  FOR  DEPARTMENT  OF  DEFEND  INSPECTOR  GENERAL 

ATTN:  PATRICIA  BRANNIN 

FROM:  AFOSR/CD 

1 10  Duncan  Ave,  Room  B1 15 
Bolling  AFB,  DC  20332-8050 

SUBJECT:  Commcnte  and  Statement  of  Concctive 

IniHative  Protrom.  Nq.  M(fY 

'■  ^*,*^TDircctive  7650J.  Follow  up  on  Geneml  Accounting  Office.  DoD  Inspector  Geneial  and 

Internal  Audit  Reports,  September  5, 1989 

b  DoD  Directive  5010.38,  ‘Internal  Management  Control  Program,”  Apnl  14, 1987 
c!  Office  of  Management  and  Budget  (OMB)  Circular  A-123,  Revised,  “Management 
Accountability  and  Control,”  June  21, 1995  .  ,  i  * 

d.  Air  Force  Policy  Directive  61-1,  August  31,  1993,  Scientific  Research  and  Development, 
“Management  of  Science  and  Technology” 

e.  Air  Force  Instruction  65-201,  July  1,  1994,  Financial  Management,  “Intemal  Management 

Controls Prognun”  -  ,  * 

f.  Air  Force  InstrucUon  65-201/AFMC  Supplement  I,  April  17.  1995.  Fmancial  Management 
“Internal  Management  Controls  Program” 

g.  Air  Force  Policy  Directive  65-2,  April  1,  1997,  Financial  Management,  “Management  Control 
Program” 

2  Commente-  The  Air  Force  Office  of  Scientific  Reseawdi  (AFOSR)  accepts  Ac  findings  tot  the 

Paymmt  Pool  Agreements,  Air  Force  Office  of  Scientific  Research  (AFOSR).  March  12, 1996. 

■j  nf  rvimrtive  Actions'  AFOSR  will  expand  its  current  Intemal  Management  Control 

oJte  .£n«.  lo  »tp»ogNpli!  lA-l*  ma  ■»/ 

mission  of  AFOSR  and  management  control  authority  delegated  by  Air  Force  Matcnel  Comma  . 


ROBERT  L.  HERKLOTJIr 


ROBERT  L.HERKLO' 
Colonel,  USAF 
Deputy  Director 


cc: 

AFRL/CC 

S  AF/AQR  (Dr.  Walter  Jones) 
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OFnce  Of  THE  ASStSTANT  SECRETARY  15  July  1997 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING 
OFFICE  OF  THE  INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 


FROM:  SAF/AQR 

1919  South  Eads  Street,  Suite  100 
Arlington,  VA  22202-3053 

SUBJECT:  Audit  Report  on  University  Research  Initiative  Program  (Project  No.  6AB-0077) 

This  is  in  reply  to  your  rhemorandum  requesting  the  Assistant  Secretary  of  the  Air  Force 
(Financial  Management  and  Comptroller)  to  provide  comments  on  subject  report. 


The  Air  Force  accepts  the  findings  that  the  Air  Force  Office  of  Scientific  Research 
(AFOSR)  policies  and  procedures  for  selecting,  awarding,  and  administering  University  Research 

Initiative  Program  grants  were  generaUy  effective.  The  Air  Force  accepB  that  the  DoD^^ 

verified  the  previous  findings  of  the  Air  Force  AudU  Agency  Installation  Report  No.  50296028. 
“Administration  of  Advance  Payment  Pool  Agreements,  Air  Force  Office  of  Scientific  Research 
(AFOSR)”  March  12, 1996. 

In  response  to  the  recommendations  of  the  subject  report,  I  am  directing  AFOSR  to 
expand  its  current  Internal  Management  Control  Program  (IMCP)  for  Management  of  Science 
andTcchnology.  AFOSR’s  expanded  IMCP  wiU  be  based  on  the  following  references,  and 
updates  thereafter,  as  they  pertain  to  the  mission  of  AFOSR  and  management  control  authority 
delegated  by  Air  Force  Materiel  Command: 

a.  DoD  Directive  7650.3,  Follow  up  on  General  Accounting  Office,  DoD  Inspector  General 
and  Internal  Audit  Reports,  September  5, 1989, 

b.  DoD  Directive  5010.38,  “Internal  Management  Control  Program,”  Aprd  14, 1987. 

c.  Office  of  Management  and  Budget  (OMB)  Circular  A-123,  Revised,  “Management 
Accountability  and  Control.”  June  21, 1995. 

d.  Air  Force  Policy  Directive  61-1.  August  31. 1993,  ScienUfic  Research  and  Development, 
“Management  of  Science  and  Technology.” 

c.  Air  Force  Instruction  65-201,  July  1, 1994,  Financial  Management,  “Internal 
Management  Controls  Program.” 
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f.  Air  Force  Instnictioo  65-201/AFMC  Supplement  I,  April  17, 1995,  Financial 
Management,  **lDtemal Management  Controls  Program” 

g.  Air  Force  Policy  Directive  65-2,  April  1, 1997.  Financial  Management,  •‘Management 
Control  Program.” 

In  summary,  the  Director  of  AFOSR  will  establish  the  required  Air  Force  management 
control  program  and  provide  training  to  personnel  responsible  for  the  management  control 
program,  as  recommended  in  the  subject  DoD  Inspector  General  report 


HELMUT  HELLWIG 

Deputy  Assistant  Secretary 

(Science,  Technology  and  Engineering) 
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This  report  was  prepared  by  the  Acquisition  Management  Directorate, 
Office  of  the  Assistant  Inspector  General  for  Auditing,  DoD. 

Patricia  A.  Brannin 
Raymond  A.  Spencer 
Roger  H.  Florence 
Dora  Yvonne  Lee 
Herbert  L.  Braun 
Gary  B.  Dutton 
Sandra  S.  Morrell 
Sonya  M.  Mercurius 
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